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ABSTRACT 



Qualitative inquiry is rich in personal interaction between 
participant and researcher. This research seeks to consider the issues 
related to conducting qualitative inquiries in online settings and the 
implications for using electronic means in interview and observation. 
Consideration needs to be given to the ramifications of online interview 
techniques. "Face- to-f ace" may not be "face-to-face" in online environments 
but may be only available in other synchronous forms, such as telephone or 
online discussion. Implications for electronic discussion formats for 
conducting interviews are also discussed. Triangulation, in the most accepted 
definition, then, may be implausible due to the fact that the interview or 
observational techniques are not logistically feasible or probable. 
Observation in natural settings is foundational to the integrity of the 
observation but should be investigated for workability as well as rectitude 
(Creswell, 1998). Exploration into the implications for observation by 
electronic means should be conducted to determine the trustworthiness of this 
new form of data collection. The methodology issues discussed in this paper 
are those that relate to the interview and observational legs of 
triangulation in conjunction with possible avenues for document analysis. 
Various strategies for testing online qualitative inquiries are considered 
and discussed related to their potential value to the inquiry process. The 
inquiry in this case centers on methodological implications for data 
collection and their impact on the qualitative process in online situations 
or by electronic means. (Contains 33 references.) (Author/AEF) 
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Abstract 

Qualii-. live inquiry is rich in personal interaction between participant and researcher . The researcher is 
an instrume, ; Creswell , 1998) and the participant an active sharer in the process and in some cases the final 
interpretation of the inferences, as well The qualitative methodological processes, particularly those considered the 
ihree legs of triangulation - interview, observation and document analysis - vary in the degree and type of 
communicative aspects in specific contexts. The various methods used in collecting and interpreting qualitative data 
are best used together - multiple methods . (Denzin and Lincoln, 1998; Creswell, 1998) Using more than one gives 
the researcher an avenue to corroborate findings and add a deeper dimension to the study. For example, the 
probing of the interview combined with the interpretation of reports and the interaction of an observation form a 
more complete picture from which themes and issues emerge, allowing the researcher to tell the story of the 
participants more accurately and completely . (Denzin and Lincoln, 1998) Will the story be affected with the advent 
of new technological methods of not only collecting the data, but the data itself? 

This discussion seeks to consider the issues related to conducting qualitative inquiries in online settings 
and the implications for using electronic means in interview and observation. Consideration needs to be given to 
the ramifications of online interview techniques. “Face-to-face” may not be ‘ face-to-face " in online environments 
but may be only available other synchronous forms, such as, telephone or online discussion . The implications for 
electronic discussion formats for conducting interviews will also be discussed . 

Triangulation, in the most accepted definition, then, may be implausible due to the fact that the interview or 
observational techniques are not logistically feasible or probable. Observation in natural settings is foundational 
to the integrity of the observation but should be investigated for workability as well as rectitude. (Creswell, 1998). 
Exploration into the implications for observation by electronic means should be conducted to determine the 
trustworthiness of this new form of data collection . 

The methodology issues discussed here are those that relate to the interview and observational legs of 
triangulation in conjunction with possible avenues for document analysis. Various strategies for testing online 
qualitative inquires will be considered and discussed related to their potential value to the inquiry process. The 
inquiry in this case centers on methodological implications for data collection and their impact on the qualitative 
process in online situations or by electronic means. 




Introduction 

There are a growing number of research studies being conducted on distance education, in particular, online 
learning environments (Schlough & Bhuripanyo, 1998; Hengni, 1998;Thompson and Nay, 1999; French, 1999; 
Moon, 1998; Mclssac, et. al. , 1999; Truman-Davis and Hartman, 1998; Gunawardena, et. al., 1998; Levin and Ben- 
Jacob, 1998; Hiltz, 1998; Chen & Mashhadi, 1998; Jannasch-Pennell, et. al., 1999; McFerrin, 1999; Donaldson and 
Tomson, 1999; Yong, 1998). Online or web-based course delivery is becoming more and more prevalent as an 
integral part of today’s college curriculum. Instructors and university professors are providing more web support 
and universities are soliciting students to participate in online coursework in the “anytime, anywhere” model of 
instructional delivery. (Gladieux and Swail, 1999, Dunn, 2000). 

As a result of the increased interest in research in the area of online learning, there has been a subsequent 
increase in the interest in the use of qualitative methods to gain a deeper understanding of online learning 
environments. In particular, there have been a large number of studies which examined, qualitatively, the student 
perceptions of online learning experiences (e.g., McNeil & Robin, 2000; Mclsaac, et.al., 2000). However, there has 
not yet been adequate exploration of the use of qualitative methods in online learning environment inquiry. 

In an effort to understand student’s perceptions of their learning in this emerging environment, much 
research is being conducted in connection to online learning. Among the current topics of interest are studies which 
compare online to traditional delivery of the same course material; effectiveness of learning; student perceptions; 
fostering collaboration; and various course delivery models, .(e.g., Diaz and Cartnal, 2000; Ryan, 2000). 
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Online learning has both its advocates and its detractors. These new technologies in instructional delivery 
carry both support for and definite opinions against online learning environments and their potential effect(s) on 
learning. If learning is at the heart of our choices regarding instructional delivery options, researchers and educators 
are obligated to fully understand student experiences in many types of learning situations. Student perceptions and 
experiences should be priorities in the design process and should be strongly considered prior to media selection 
decisions. It is difficult not to design for the "delivery method d'jour", however no matter how glizty or well-funded 
online education options may be, their impact on learners' experiences as understood through qualitative inquiry 
should be carefully considered by all practicing instructional designers. 

In an attempt to understand the culture of online-learning, qualitative methodology is a natural choice with 
regard to research design. The tenets of phenomenology (Denzin and Lincoln, 1998) lead us to deeper 
understandings of the experiences of the participants and the interpretation that informs our understandings. 
Phenomenological research assumptions carry with them the importance of the voice of the participant, their 
descriptions of their experiences, their reflections on their lived experiences. (Creswell, 1998; Moustakas, 1994). In 
considering online learning environments, these experiences are critical in an attempt to gain deeper understandings. 
This paper is an attempt to discuss some of the potential implications of doing qualitative research with online 
populations. 

Distance Education - Attitudes and Perceptions 

David Noble’s (2000) outcry against online learning and technology is largely aimed at the politics of 
technology. He argues that the technology itself is indeed value-laden and the demand for online education is non- 
existent, created by university administrators. (Young, 2000) He suggests that the motivations for online learning 
are nothing more than greed on the part of university administrators and should be resisted by faculty in the same 
ways that Luddites resisted the new technologies that they believed would put their own children out of future work. 
Others (Bowers, 1999; Woody, 1999) also argue that while the Internet is an excellent dispenser of information, 
education is not merely the acquisition of information. By making learning primarily about the acquisition of 
information, there is the potential to commodify higher education— learning becomes a product for sale rather than 
the experience of learning and growth. Like Noble, many are concerned for the financial emphasis placed on 
online learning environments with a lack of focus on “learning”. (Noble, 1998; Woody, 1998). 

Many supporters of online education (Dunn, 2000; Dede; 1997). envision a future where students will come 
to universities only for the social aspects of higher education, such as sports or other non-academic activities. In a 
report prepared by the College Board, predictions about the future of online learning environments included the 
demise of the university, as we know it, within 30 years. (Gladieux and Swail, 1999;Dunn, 2000). Gladieux and 
Swail go on to cite examples of complete degree programs offered by “traditional” universities online and refers to 
them as leaders in the electronic market. While considerations need to be made about the access to these new 
technologies and the target populations who benefit from technological advancement, other issues should be 
considered as well. The need exists for a qualitative examination of online instruction. Collaboration, social 
interaction and relationship building activities that exist in face-to-face instruction have been addressed somewhat in 
design literature 

Qualitative methodologies are social at the core (Creswell, 1998;Denzin and Lincoln, 1998). There has 
always been an understanding that part of the qualitative inquiry experience is "being in the field." (Wolcott, 1995). 
Doing fieldwork, in which the inquirer worked in a natural setting to collect stories by observing, examining 
documents, and interviewing indigenous populations, has always been a cornerstone of the qualitative research 
experience. (Lincoln and Guba, 1985). What is the implication of doing qualitative research in online settings? 
Where is the field? Where is it located? Is it electronic? Collecting data at a distance begs the question - can 
qualitative research occur in the exploration of online learning and if so, how will that experience be defined? Will 
qualitative research lose a certain social quality when we translate these methods from naturalistic inquiry (Lincoln 
& Guba) into e-inquiry? Will the researchers enjoy the experience of doing qualitative research in an e-field as 
much as they enjoy being in a physical field? Will there be some important essence of the stories that are lost by 
doing qualitative research electronically? Is there any way to do qualitative research among online populations in a 
face-to-face mode and what is gained or lost in this choice? And what of the qualitative methods themselves? Will 
the traditionally separate methods of interview, observation and document analysis become blurred in online 
settings? If an interview is conducted in a chat mode, is it a document to be analyzed or an interview? Clearly we 
cannot answer all of these questions in any complete way here, we raise them as serious considerations for those 
engaged in qualitative research among online populations as well as for those who supervise students who wish to 
pursue qualitative inquiry projects in online education. 
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Qualitative Research Epistemological Viewpoints 

Qualitative research is conducted in an effort to understand the experiences and attitudes of people in 
contextually bound settings. Purposes of qualitative research include inquiry in natural rather than contrived settings 
and the understanding the ontological viewpoints of the participants. Perspective, perception and experience are 
epistemological notions that are sought in the interpretation and understanding of specific contexts or situations. 
Because we wish to understand perceptions, experiences, and perspectives of learners in online environments, there 
is a viable match between qualitative inquiry and online learning environments. However, online qualitative inquiry 
methodology has not been clearly delineated or explicated nor is it definitively prescribed in the qualitative inquiry 
literature. In some ways we have gotten ahead of ourselves. We are using qualitative inquiry because it appears to 
be an epistemological match with our goals for understanding online learning environments prior to truly exploring 
the impacts of this application on the field of qualitative inquiry and the underlying epistemological foundations on 
which that discipline is built. 

The discussion that follows is an overview of the three most common qualitative data collection methods 
(interviewing, observations, and document analysis) and their relevance to the understanding of online learning 
environments. We explore some of the implications of using qualitative methods and base these implications on our 
own experiences conducting qualitative studies in online environments. 

Qualitative Methodology 

Interviewing 

Interviews and observations are used as key methods of recording people's experiences, perceptions and 
attitudes in qualitative inquiry. (Creswell, 1998). Interviews range in type and length and are used for different 
purposes but are present in virtually all qualitative traditions. Interviewees are often selected utilizing purposeful 
sampling processes and are contacted in many different ways. Telephone interviews are conducted when access to 
participants in a face-to-face environment are hindered in some way. (Bruce, 1979; de Leeuw, 1992). Telephone 
interviews carry the advantage that the participant can be interviewed at a remote location saving travel time and 
money, but the researcher loses the opportunity to observe nonverbal communicative actions. Focus group 
interviews are used in situations where the interaction among participants is determined to be beneficial for the 
inquiry. Ideal interview situations are personal, face-to-face contexts where the interviewer and the interviewee 
have the opportunity to interact in an open atmosphere and establish rapport (Fontana & Frey, 1994). Typically, 
qualitative interviews are recorded either by audio or video and are transcribed for the analysis of data. There are a 
number of ways of approaching interviews from open-ended to highly structured interview protocols. 

Computer-mediated interviewing (CMI) techniques add quite another dimension. Media utilized to 
conducte CMI's include synchronous chat, email and discussion forums. As with telephone interviews, in CMI, 
face-to-face contact is lost. Voice tone and inflection, hesitation or eagerness, and other audible indications that lead 
to deeper understandings that are not possible through CMI are lost. While not without benefit, interviews 
conducted via chat may arouse questions regarding the conveyance of meaning since voice and body language are 
absent. On the other hand, little is left unquestioned in terms of specific data because it is necessary to ask for 
continual clarification from the participant throughout the CMI. 



Face-to-Face 


Telephone 


Computer-mediated 


Visibility of facial 
expressions 


Absence of facial 

expressions and body 
language 


Absence of facial 

expressions and body 
language 


Personal qualities of 
establishing rapport with 
the participant 


Voice tone assistance in 
establishing rapport 


Absence of personal 
contact to aid in 
establishing relationship 
with participant s). 


Communication is more 
natural to most people in 
conversation form 


Conversational tones are 
possible 




Travel time required 


Travel time saved 


Travel time saved 


Transciption costs 

incurred 


Transcription costs 

incurred 


Transcription costs 

saved 


Travel costs incurred 


Telephone costs may be 


Typically no or low cost 
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incurred depending on 
type of long distance 
utilized 


via internet hookup 


Opportunities for on-site 
experiences are 

preserved 


Opportunities for on-site 
experiences lost 


Opportunities for on-site 
experiences lost 


Ability to interview a 
limited number of 

respondents 


Ability to interview 
more respondents 


Ability to interview 
many respondents 

including those abroad, 
disabled, elderly, 

housebound 



Table 1: Comparison of interview data collection modalities 

It is possible to see from Table 1 a comparison of the advantages and disadvantages of the three possible media for 
conducting qualitative interviews. Obviously the primary advantage of using the telephone or the CMI options are 
resource related, e.g., time and money to travel to remote sites. However, there is no clear indication of precisely 
how much is lost in terms of the deep understanding of the stories being told by the respondents by losing their 
affective, interpersonal, voice and body reactions to questions. It is not clear how much data may be lost by the very 
different type of rapport that is established online or by telephone and it is almost impossible to quantify or qualify 
this loss because qualitative data is so individualized that there is no easy comparison study that can be conducted to 
establish what is truly lost and/or gained in these three environments. We believe that these decisions are likely to 
be made too often because of cost considerations. Rather than seeking additional resources to allow the researcher 
to be in the natural setting, the breadth of respondents will be valued and the choices will be made to utilize cheaper 
methods than traveling to the site. It is the same principle as we use in the assignation of media in instruction. We 
ought not use a media merely because it is available, new, or glitzy— or well funded, but rather because it is what is 
called for in the instructional design based on the assessment, strategy selection, goals and objectives for the learner. 
In this same way, we should select telephone or CMI options for collecting qualitative data because it is supported 
by the research question . Methodological choices should not be made for financial reasons alone any more than 
they should be made for convenience reasons alone-thus the negative perception of samples of convenience studies. 
Instead we should allow our research question to drive our interview method choices among face-to-face, telephone, 
and CMI. 

Observations 

Qualitative observations (Adler & Adler, 1994) elicit some of the same issues as interviews, but the 
concerns here are even more pronounced. How is it possible to "observe" an online class? We know what it means 
to observe a face-to-face classroom, and we know how to design an observation instrument to assist us in focusing 
on the proper interactions, experiences, cultures, and environmental cues. However, we don't know what any of this 
means in the online environment. Do we observe individuals at their machines in their home space? Do we observe 
the class as it interacts online? Is the electronic space the actual classroom? What sorts of things are we looking for, 
what type of instrument will help us to focus on those things? Participant observations where the researcher is part 
of the culture in which the observation is being conducted, as is the case with ethnographic studies, leave the role of 
the researcher even more undefined in online environments. It is important to consider the practical as well as the 
theoretical and philosophical aspects of participant observations in online class settings. 

In our experiences this is perhaps one of the most difficult things to handle in online learning environment 
inquiry. Because we do have certain "scripts" (mental or physical) which guide our interview protocols, the 
interview procedures are still somewhat familiar to us in electronic media. Although the social nature of the 
interview is significantly altered, the pragmatics of how to go about it are still relatively similar-that is to set out as 
series of questions (structured, unstructured, semi-structured) and begin conversing with respondents. But how do 
we go about really observing the online learning environment? Is it even possible? Here is an even more evident 
exposure of the social nature of qualitative inquiry (Wolcott, 1995). Here we have a real rub, what sort of 
observational techniques should we use, can we rely on the old checklists for observations that we had used 
traditionally? Do we have to shift foci in the same ways that we do in traditional settings? Unfortunately, our 
experience raised more questions than answers in this regard. We attempted to "observe" online settings by reading 
through the ongoing synchronous exchanges such as chats and asynchronous discussion forum communication. In 
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the end, our experience was so limited in this regard, that there is little we can really say by way of overall 
impressions or guidance. Rather we can merely raise these questions for careful —even cautious— consideration by 
those embarking in online learning environment inquiry. 



Document Analysis 

Document analysis (Hodder, 1994) is perhaps the least difficult qualitative data issue to be resolved. Since 
there are fewer abstractions that are important (e.g., voice tone, culture of a classroom, etc.) in document analysis, 
the issues are a little less complicated. Clearly online course materials as with face-to-face course materials are 
prime candidates for document analysis in such studies. In the same way, student materials are also documents. But 
in the online environment, there are typically many more "printed” resources than there may be in the face-to-face 
environment. What is to be considered a document? Is the chat that the students conduct on regular Monday 
meeting times, for example, to be considered documentary, observation, or interview data? Documents typically 
included in qualitative research for data analysis appear in the form of written text. Written text assumes a purpose 
depending on the author and the intended use of the document. Creswell (1998) distinguishes between texts written 
as records of information, i.e., public documents, reports, contracts and others such as diaries or journals, written for 
personal use. Journals, logs and diaries are sometimes requested by the researcher as a way for participants to keep 
track of and reflect on their activities in a particular situation or context. 

In online learning environments, information is presented in the form of written text in several aspects 
including (but not limited to) synchronous chat, discussion forum, project submissions, email communication and 
written reports or summaries. As these class activities are utilized in ways that intend to mirror activities typically 
integral to face-to-face learning environments, in which case they would not be considered in document analysis, but 
rather as observation or interview data. In what ways does this change the nature of data collected. In the case of 
document analysis, wherever this cautious distinction is drawn and there is care to maintain authenticity in the 
documents themselves, it is likely that, in this one area, there is the least impact of the switch from face-to-face to 
online inquiry subject. This is primarily because there is inherently less "socialness" to the procedures within most 
document analyses even within the qualitative tradition. It is more anthropological in nature, uncovering the 
cultures and understandings of authors as a result of careful dissection of documented data. 

Conclusion and a Note of Caution 

With the growing interest in online education and the vast difference in teaching and learning between 
online and face-to-face learning environments, it is becoming more and more necessary to gain a better 
understanding of the experiences students face in new distance education endeavors of many institutions of higher 
education. While there is little research available in this area, we believe qualitative methods in seeking this 
information, are not only appropriate but necessary in developing a greater understanding of student experiences. 
However, the employ of qualitative methods founded on assumptions of the social nature of inquiry can only be 
used with great caution. It is essential that as we continue to examine online learning environments in qualitative 
ways that we carefully reflect on the experience of doing online interviews, online observations and electronic 
document analysis and try to capture how we see these experiences as differing from previous experiences in 
qualitative research. 

It is essential that all good qualitative research have certain hallmarks of quality, and in one final note, we 
wish to emphasize the importance of disclosing one's researcher identity. Even more so than in many other areas, 
the online revolution has recently turned into an online opinion war with many folks lining up on both sides, pro 
(Dede, 1997 ) and con (Noble, 1998). Because the online learning craze is likely to affect our own lives and 
livelihoods, it is essential that we are clear with all readers of precisely our own feelings and biases with regard to 
this new innovation in honest, up front ways. 
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